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Abstract—Normalization of design rules and guidelines in English lan- 
guage tests to help future instructors and assessment developers in the field of 
examination of English as a foreign language. have not been defined sufficient- 
ly until present days 

In fact, there are several experiences and investigations related to the user’s 
experiences in the handling of computer exams for learning a foreign language 
that have been evaluated in, for and through different experimental interactive 
digital environments. 

These interactive scenarios oriented to the user experience, UX in the ubiq- 
uitous devices, have allowed to learn technological, functional and design as- 
pects that will be necessary in the future to establish the standards in design and 
vision for language tests in its online environment. 

The lack of effective and realistic regulations has become a functional re- 
quirement for the progress of research on digital environments adapted to the 
needs of competence assessment and more specifically, to the learning envi- 
ronment of foreign language (FL). 

Proliferation of on-line tests of language certification of ls with a structural 
corpus depends on where there are not really functional and formal guidelines 
that allow evaluating their quality and adequacy from the experience of the user 
and according to the level of learning competences acquired in a foreign lan- 
guage. 

This article describes aspects of design that should be taken into account 
when defining an online test of languages in a ubiquitous device that will mark 
verifiable guidelines of a general nature and affect the evaluation task in the 
formation of languages that need to be checked and evaluated periodically. 


Keywords—m-learning, English as a Foreign Language, Mobile-Assisted Lan- 
guage Learning (MALL), User Experience (UX), normalization, online test, de- 
sign 
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1 Introduction 


Language exams for learning have evolved in the last 20 years due to the spectacu- 
lar increase in the need for language certifications worldwide that currently directly 
affect the educational and social systems of each country. The mobility of people at a 
global level implies the need to have a communication channel - a language that al- 
lows them to develop socially in countries other than their own. The acquisition of a 
foreign language is turning into a vital aspect for social and communicative integra- 
tion that in many cases is being regulated to verify its effectivity. Therefore, there is 
currently a proliferation of certified exams that verify the levels of knowledge or 
skills of the person from a regulated point of view.at educational and professional 
level 

The current situation in the field of investigations focused on the evaluation ap- 
plied to the knowledge of a foreign language have been developed in three areas: 1, 
The external evaluation carried out by international evaluating entities of a public or 
private nature (Educational Testing Service, Cambridge Board of Examinations™, 
Trinity College Examinations™, etc.), 2- The computer-assisted examinations orga- 
nized by educational institutions for the improvement of students’ competences. 3- 
The development of new on-line evaluation methods based on ubiquitous devices that 
facilitate learning methods based on m-learning and b-learning. 

Most of the exams or tests have the same structure and work with the same compe- 
tence parameters of and skills assessment that directly affect the learning of a lan- 
guage such as grammar, vocabulary, conversation, etc. In fact, learning of a foreign 
language and its certification maintains many traditional schemes that have been im- 
plemented in computer-assisted language learning and more specifically in Mobile- 
Assisted Language Learning (MALL). Current increase in application for the learning 
of ad-hoc languages based on collaborative environments, and collaboration on mo- 
bile devices that includes test-type assessment tests which allow both qualitative or 
quantitative evaluation of progress of each learner. Burston also points out that 75% 
of all MALL Apps are focused on individual learning as opposed to traditional teach- 
er-centred, instruction [1], [2]. 

Tests or exams are usually implemented on the digital educational platforms as a 
part of the training process and verification of learning about a specific topic. The 
process of creation and implementation of a new tests is generated in a traditional way 
by selecting a specific type of questions (questions with long and / or short answer, 
true / false, selection of options, etc.), determination of a quantitative or qualitative 
qualification format and finally by a way of collecting the overall mark of all the tests 
carried out to obtain a total or partial assessment that serves as a complement to the 
student's learning [3]. 

The creation of a conventional on-line test establishes the development of a back- 
end or internal platform where all the technical and formal aspects of exams creation 
are generated and implemented, and a front-end external platform oriented to the 
student that is used in a controlled and monitored way visualized in different devices 


[4]. 
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The visualization of the front-office of the foreign language test or the certification 
test is done within the final stages of the development process in the digital environ- 
ment by following the administration agency’s guidelines which in turn are set by the 
navigation philosophy of the digital platform using the same data source and type of 
interface that was formally adapted to digital content depending on the size of a de- 
vice such as tablet, PC and smartphone [5]. As it can be summarized, the process 
usually begins by pen-and-paper, then it goes online for large screen devices and 
finally, if so desired, it will be finally adapted to mobile devices. This is a debatable 
approach obviously but it fits perfectly in the principle of economy since the creation 
of specifically aimed (to mobile devices) of testing platforms can dramatically in- 
crease the complexity and thus the price. 


2 Interaction methods in management and access of 
information in language certification of on-line tests 


Due to the evolution of capacity and functionality of ubiquitous devices, an infor- 
mation access framework is needed for the design process: The use of interactions 
with the user that are increasingly natural and combined with more than one channel 
as is the case of applications based on multimodal interaction methods creates a better 
experience for the test candidate. In the case of multiple choice exams, multimodality 
will be a future line of research that will mark the evolution in the development of 
navigation processes on a specific exam such as the case of language testing [6]. 

Multimodal interaction is a process in which the device and the user are able to 
perform a joint interaction — whether auditory, visual, tactile or kinaesthetic, at any 
place and any time in an accessible way. The study of levels of multimodal integra- 
tion is related to the technological use of connected interaction [7]. 

This method of interaction allows us to simulate human communication where, for 
example, verbal communication is a simultaneous set of perceptions obtained from 
sight, hearing, gestures, etc. It permits us to have an idea about the situation, the 
context in which we speak, recognize the interlocutor and respond to the message 
received. Multimodal interfaces can reduce the recognition of errors by the user be- 
cause the user can confirm data in multimodal way and through several channels. 
Combining entries by different modalities can improve the recognition of a task (see 
Figure 1). Through multimodal interaction, users can determine a mode or different 
modes of interaction needed to be used during the process of accessing information by 
either keyboard, mouse, pencil, touch screens, voice, etc, on an interface specially 
developed for it [8]. In the case of test development for language certification exams 
on a ubiquitous device, the most common interaction option is a touch screen used by 
many applications for tablets in the educational field. 
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Fig. 1. Input/Out data channels oriented to test on-line design for ELF (Magal-Royo, 2018) 


In recent years the term multimodal in computer language level of development 
has been revitalized due to the technological possibilities offered by most devices 
currently in use such as mobile phones, digital television, computers, etc. The ad- 
vances of the Consortium for web accessibility called W3C, currently aims to tech- 
nologies and interactive development processes accessible over the network. The 
standards generated by W3C for the mobile web are the basis of this type of interac- 
tion since they comply with all accessibility requirements not only at the digital con- 


tent level but also with navigation guidelines. 
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W3C provides the technologies that facilitate interaction through hearing, sight and 
touch based on languages such as Voice XML, HXHTML and ENMA, the latter act- 
ing as a communication mechanism within a multimodal system such as in a language 
certification online test. 

Therefore, in the future the use of multimodality in the development of language 
testing will improve communication between the user and the device and will open 
the doors to a more natural handling of digital content created for this purpose. 

In the field of technical and functional feasibility of developing multimodal inter- 
faces for language learning from the point of view of some of the basic skills, various 
experiences have emerged in the University access language tests as in the case of 
Spain [9], [10], [11]. 

In conclusion , we can say that the process of control of interactivity by the user 
will become a key element in the design and formalization of a certification test. Fol- 
lowing criteria have to be taken into account in the process of creating applications 
with the specific purpose of assessing the knowledge of a foreign language: 


— Configuration of communication channels and transmission control of information 
on digital media. 

— Multimodal interactive digital environments controlled by the user. 

— Technical accessibility criteria based on W3C that allow access to people with 
visual and / or cognitive disability. 

— Creation of a guided navigation according to the needs of the user. 


3 User experience (UX) as a determining factor in the creation 
of a standardization applied to an on-line test 


UX performance demands the User-Centered Design (UCD) to developed systems 
that users need. The aim of UCD in technology is to act as an effective interface for 
delivering pleasant experiences to users when interacting with computers. It is a flexi- 
ble interactive design methodology, a philosophy and a discipline that understands 
and analyses how people use computers, objects and systems to propose, accessible, 
aesthetically pleasing and easy-to-use interfaces [12]. User experience in Human 
Computer Interaction (HCI) is described by the ISO 9241-210: 2010 normative as 
person's perceptions and responses resulting from the use and / or anticipated use of a 
product, system or service [13]. 

User experience should therefore be the central axis that marks formal and norma- 
tive aspects of foreign language assessment tests since it would allow a more effective 
interaction [14]. At present, several studies investigate this problem from three differ- 
ent points of view —from the formal aspect of the content, from the point of function- 
ality and usability of the test and from the point of the use of interfaces that facilitate 
efficient interactive navigation. It is precisely the development of the test by the user 
and the conditions of online access to a digital environment that needs a standardiza- 
tion. The goal is to improve and deepen research aimed at the user experience (UX), 
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compared to the technology that is being used among other things because the user 
already has experience and knowledge in the digital medium [15], [16]. 

User experience (UX) includes a multidimensional concept and focusses more on 
human needs and the aspects of beauty, fun, pleasure, and personal growth than on the 
value of the product or instrument used [17], [18]. In fact, in the design of tests for 
language learning, it is convenient to consider the identified elements when develop- 
ing and evaluating designs and provide clear instructions on how to perform mobile 
collaborative activities [19]. 

Therefore, it implies the need for standardization from the user’s point of view who 
performs a language knowledge test that helps the creators of this type of exam un- 
derstand the importance of ergonomic, visual and usability factors as opposed to the 
control of technical aspects derived from the technologies on which the test is imple- 
mented on a digital level. The fact that on-line tests are carried out in one device over 
another should not influence the fundamental objective that it would be to provide 
evidence of language knowledge with the same degree of difficulty once the test taker 
becomes familiar with the way of interactivity and navigability [20]. That means that 
the bias introduced by the way of delivery should be null, and the student should be 
able to get similar scores no matter the way of delivery (whether paper, traditional 
computer or mobile device) is used. Therefore, the aspects related to the user experi- 
ence in the handling of a test for the knowledge of languages focus on the following 
aspects: 


— Feeling of predictability in application management. 

— Sensation of control of the application. 

— Effectiveness in execution tasks. 

— Safety and reliability of control data (personal data and results). 

— Access to follow up information 

— Functional Aesthetics 

— Easily recognizable interface based on technical and graphic characteristics of the 
device. 


4 Research of on-line tests for foreign language (FL) 
researchers in ubiquitous devices 


Since more than 10 years ago, there have been pilot experiences that have investi- 
gated the use of ubiquitous devices in the learning and development of computer- 
assisted assessments for language learning. Most of experiences initially extrapolated 
conventional analogue schemes and scenarios (paper examinations) into digital envi- 
ronments on desktop computers that were connected via intranet to an educational 
network and / or were connected online to the Internet. The creation of these digital 
on-line exams posed several challenges. They not only focused on the creation of the 
exam or the test itself on the applications available at the time, but also posed a great- 
er challenge depending on the involved technology and user’s environment 
knowledge. 
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Thus, the user should become accustomed to the adaptability to digital medium at a 
functional and procedural level as well as to other aspects such as visual treatment of 
digital audiovisual content, interaction between examination and user, method of 
quantitative and qualitative evaluation of the results obtained, etc. [21]. 

Different proposals have been presented in various research projects such as 
OPENPAU, PAULEX-Universitas, and SELECTOR (where the impact on the use of 
the computer and mobile phones in evaluation and learning processes of a foreign 
language was verified [22]). While other projects have focused on the user’s experi- 
ence and also on the interactive environments oriented towards the development of 
on-line language assessment tests. These works have made possible to understand 
adjusting and customizing user's environment during on-line test through the use of 
multimodal interfaces, adapted digital content and aesthetics based on user-oriented 
functional ergonomics. 

Knowledge acquired so far has allowed us to analyze a possible normalization con- 
text based on a series of parameters oriented towards user experience and access to 
information in a multimodal manner that will have to be analyzed in future in various 
educational contexts. 


4.1 Standardization in the implementation of a foreign language knowledge 
test using ubiquitous devices. 


The standardization concept in the creation of online examination tests using ubiq- 
uitous devices needs to be considered from four points of view —1- The theoretical 
corpus of the evaluation test that is traditionally based on the verification of the 
knowledge of the in foreign language skills (writing, grammar, conversation, listen- 
ing, etc. ), 2- The technological corpus that encompasses technologies capable of 
supporting interactive access structures to each of the evaluable items. That fulfils as 
main characteristics the ubiquity in the access to information, the interoperability of 
the data of the digital environment between ubiquitous devices and the technical ac- 
cessibility to the information shown; 3- The functional corpus that determines the 
process of development of the tests including the interactive functional design, the 
execution of the test and the analysis of the results. This section would include the 
method of analysis, control and verification of student data that will be processed and 
displayed on an interactive environment, 4- The corpus of user experience focused on 
the access parameters that affect the interaction in direct, multimodal and collabora- 
tive way according to the needs of the user [23]. 

All these aspects must be treated and analysed in the conceptual and functional de- 
velopment of a language assessment test since in most cases development is consid- 
ered as a mere migration between technological scenarios on a specific digital device 
(Figure 2). 

Normalization implies the control or verification of the naturalization process by 
the test taker and should be taken into account when planning an international lan- 
guage test because it improves the systemic aspect of the functionality regardless of 
the theoretical corpus applied to assess the language. That obviously means, that it has 
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Fig. 2. Scenario of the factors related to the creation of on-line exams for language learning. 
(Magal-Royo, 2016) 


no or very little effect on the test candidate’s performance. Therefore, it is important 
to note that the aspects to be taken into account in the development of a certifiable 
language test should work very much in the interactivity aspects with the digital envi- 
ronment from the user experience. by taking into account that the technology and the 
device is a mere channel of representation that allows us to have the information in 
the moment. That obviously implies, that normalization and the above null effect of 
technology on the student’s performance is actually the real goal in developing the 
test to be used on a mobile device. If we propose the systemic regulation based on the 
use of ubiquitous devices we will also find the paradigm of adaptability and use any- 
where that is currently being investigated from the point of view of the active learning 
methodologies throughout life. Each time the device is less important and it is the 
type of man-machine communication and the experience that is obtained from it, 
which should actually orientate the design of both the design of the software and the 
election of the hardware. 


5 Conclusions 


The need to maximize the user experience during the examination process should 
be the cornerstone on which the parameters related to standardization in the creation 
of an online language test should be defined. Until now, the determining factors have 
been the applied technology and the means by which the on-line examination was 
implemented and its subsequent evaluation. Nevertheless, currently the user is famil- 
iar with the digital medium and the process of conducting tests of languages certifica- 
tion both analogic and digital. 
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As a conclusion, we can say that normalization is the only way to reduce of differ- 
ences among different test takers and avoid undesired bias introduced by the capacity 
to use mobile devices. That means, when normalization will be achieved tests will 
measure what they are intended to, that is the learners’ knowledge of foreign lan- 
guage. After all, should a test measure anything else other than the student’s perfor- 
mance in that foreign language? Of course, not. 
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